Interference Mitigation in Femtocell Network Using Q-learning

The idea of femtocell have attracted lots of interest from the research community of wireless communication. Femtocells aim to increase the capacity and coverage area of the current cellular network by helping to reduce its actual cell size. However, new design challenges arise by randomly deploying the femtocells over the cellular network, hence a heterogeneous network. One of the main problems is the so-called co-tier and cross-tier interference caused by the new femtocell network layer, which is operating in the same frequency spectrum as the cellular network. Various techniques have been proposed to deal with the interference management issue in heterogeneous networks. This paper investigates the use of reinforcement learning algorithms to solve interference problems in two tier heterogeneous networks. We assume the femtocell user equipments (FUEs) and macro user equipments (MUEs) to be selfish and we try to guarantee the Quality of Service (QoS) of all users equally if possible. We formulate the power adaptation process of the FUEs and MUE to be a discrete multi agent Markov decision problem and solve it by using the well-known Q-learning algorithm. In the MDP process, each agent adapts its own transmission power by learning from the environment. Numerical results show that the distributed decision process will converge to an equilibrium and make the system more efficient.